1. Field of the Invention
The invention relates to a redundant external storage virtualization computer system. More particularly, a redundant external storage virtualization computer system that uses point-to-point serial-signal interconnects as the primary device-side IO device interconnects is disclosed.
2. Description of the Prior Art
Storage virtualization is a technology that has been used to virtualize physical storage by combining sections of physical storage devices (PSDs) into logical storage entities, herein referred to as logical media units (LMUs), that are made accessible to a host system. This technology has been used primarily in redundant arrays of independent disks (RAID) storage virtualization, which combines smaller physical storage devices into larger, fault tolerant, higher performance logical media units via RAID technology.
A Storage virtualization Controller, abbreviated SVC, is a device the primary purpose of which is to map combinations of sections of physical storage media to logical media units visible to a host system. IO requests received from the host system are parsed and interpreted and associated operations and data are translated into physical storage device IO requests. This process may be indirect with operations cached, delayed (e.g., write-back), anticipated (read-ahead), grouped, etc. to improve performance and other operational characteristics so that a host IO request may not necessarily result directly in physical storage device IO requests in a one-to-one fashion.
An External (sometimes referred to as “Stand-alone”) Storage Virtualization Controller is a Storage Virtualization Controller that connects to the host system via an IO interface and that is capable of supporting connection to devices that reside external to the host system and, otherwise, operates independently of the host.
One example of an external Storage Virtualization Controller is an external, or stand-alone, direct-access RAID controller. A RAID controller combines sections on one or multiple physical storage devices (PSDs), the combination of which is determined by the nature of a particular RAID level, to form logical media units that are contiguously addressable by a host system to which the logical media unit is made available. A single RAID controller will typically support multiple RAID levels so that different logical media units may consist of sections of PSDs combined in different ways by virtue of the different RAID levels that characterize the different units.
Another example of an external Storage Virtualization Controller is a JBOD emulation controller. A JBOD, short for “Just a Bunch of Drives”, is a set of physical DASDs that connect directly to a host system via one or more a multiple-device IO device interconnect channels. PSDs that implement point-to-point IO device interconnects to connect to the host system (e.g., Parallel ATA HDDs, Serial ATA HDDs, etc.) cannot be directly combined to form a “JBOD” system as defined above for they do not allow the connection of multiple devices directly to the IO device channel.
Another example of an external Storage Virtualization Controller is a controller for an external tape backup subsystem.
The primary motivation in configuring a pair of external storage virtualization controllers (SVCs) into a redundant pair is to allow continued, uninterrupted access to data by a host (or more than one host) even in the event of a malfunction or failure of a single SVC. This is accomplished by incorporating functionality into the SVCs that allow one controller to take over for the other in the event that the other becomes handicapped or completely incapacitated.
On the device side, this requires that both controllers are able to access all of the physical storage devices (PSDs) that are being managed by the SVCs, no matter which SVC any given PSD may initially be assigned to be managed by. On the host side, this requires that each SVC have the ability to present and make available to the host all accessible resources, including those that were originally assigned to be managed by the alternate SVC, in the event that its mate does not initially come on line or goes off line at some point (e.g., due to a malfunction/failure, maintenance operation, etc.).
A typical device-side implementation of this would be one in which device-side IO device interconnects are of the multiple-initiator, multiple-device kind (such as Fibre, Parallel SCSI), and all device-side IO device interconnects are connected to both SVCs such that either SVC can access any PSD connected on a device-side IO device interconnect. When both SVCs are on-line and operational, each PSD would be managed by one or the other SVC, typically determined by user setting or configuration. As an example, all member PSDs of a logical media unit (LMU) that consists of a RAID combination of PSDs would be managed by the particular SVC to which the logical media unit itself is assigned.
A typical host-side implementation would consist of multiple-device IO device interconnects to which the host(s) and both SVCs are connected and, for each interconnect, each SVC would present its own unique set of device IDs, to which LMUs are mapped. If a particular SVC does not come on line or goes off line, the on-line SVC presents both sets of device IDs on the host-side interconnect, its own set together with the set normally assigned to its mate, and maps LMUs to these IDs in the identical way they are mapped when both SVCs are on-line and fully operational. In this kind of implementation, no special functionality on the part of the host that switches over from one device/path to another is required to maintain access to all logical media units in the event that an SVC is not on-line. This kind of implementation is commonly referred to as “transparent” redundancy.
Redundant SVC configurations are typically divided into two categories. The first is “active-standby” in which one SVC is presenting, managing, and processing all IO requests for all logical media units in the storage virtualization subsystem (abbreviated SVS) while the other SVC simply stands by ready to take over in the event that the active SVC becomes handicapped or incapacitated. The second is “active-active” in which both SVCs are presenting, managing, and processing IO requests for the various LMUs that are present in the SVS concurrently. In active-active configurations, both SVCs are always ready to take over for the other in the event that one malfunctions, causing it to become handicapped or incapacitated. Active-active configurations typically provide better levels of performance because the resources of both SVCs (e.g., CPU time, internal bus bandwidth, etc) can be brought to bear in servicing IO requests rather than the resources of only one SVC.
Another essential element of a redundant storage virtualization system is the ability for each SVC to monitor the status of the other. Typically, this would be accomplished by implementing an inter-controller communications channel (abbreviated ICC) between the two SVCs over which they can exchange the operating status. This communications channel may be dedicated, the sole function of which is to exchange parameters and data relating to the operation of the redundant storage virtualization subsystem, or it can be one or more of the IO device interconnects, host-side or device-side, over which operational parameters and data exchange are multiplexed together with host-SVC or device-SVC IO-request-associated data on these interconnects.
Yet another important element of a redundant storage virtualization system is the ability of one SVC to completely incapacitate the other so that it can completely take over for the other SVC without interference. For example, for the surviving SVC to take on the identity of its mate, it may need to take on the device IDs that the SVC going off line originally presented on the host-side IO device interconnect, which, in turn, requires that the SVC going off line relinquish its control over those IDs.
This “incapacitation” is typically accomplished by the assertion of reset signal lines on the controller being taken off line bringing all externally connected signal lines to a pre-defined state that eliminates the possibility of interference with the surviving SVC. Interconnecting reset lines between the SVCs so that one can reset the other in this event is one common way of achieving this. Another way to accomplish this is to build in the ability of an SVC to detect when itself may be malfunctioning and “kill” itself by asserting its own reset signals (e.g., inclusion of a “watchdog” timer that will assert a reset signal should the program running on the SVC fail to poll it within a predefined interval), bringing all externally connected signal lines to a pre-defined state that eliminates the possibility of interference with the surviving SVC.
Traditionally storage virtualization has been done with Parallel SCSI or Fibre IO device interconnects as the primary device-side IO device interconnects connecting physical storage devices to the storage virtualization controller pair. Both Parallel SCSI and Fibre are multiple-device IO device interconnects. A multiple-device IO device interconnect is a form of IO device interconnect that allows multiple IO devices to be connected directly, meaning without the need for any extra off-device intervening active circuitry, to a single host system or multiple host systems (note that typical FC-ALJBODs do have off-device active circuitry, but the purpose of this circuitry is not to enable the interconnection but rather to accommodate the possibility of a failure in the DASD or a swapping out of the DASD that might cause a fatal break in the IO interconnect). Common examples of multiple-device IO device interconnects are Fibre channel FC-AL and Parallel SCSI. Multiple-device IO device interconnects share bandwidth among all hosts and all devices interconnected by the interconnects.
Please refer to FIG. 1, where a block diagram of a conventional redundant external storage virtualization computer system is illustrated. Note the interconnection of the host-side IO device interconnects that allows an SVC to take over for its mate by taking over the IO device interconnect IDs that would normally be presented onto the interconnect by its mate and mapping logical media units to these IDs in the same way its mate would. Also, note the interconnection of the device-side IO device interconnects that allow both SVCs access to all PSDs connected to the device-side IO device interconnects. In this example, a typical IO device interconnect that might be used on either host side or device side might be parallel SCSI or Fibre FC-AL, both multiple-initiator, multiple-device IO device interconnects. Therefore, both SVCs operating in target mode (i.e., device mode) are connected to a single interconnect on the host side and allow both SVCs operating in initiator mode, together with multiple devices, to be interconnected on the device side. The configuration shown in FIG. 1 suffers from the drawback that a malfunction of a single PSD, depending on the nature of the malfunction, can potentially bring down an entire device-side IO device interconnect making all other PSDs connected on the same interconnect inaccessible.
FIG. 2 diagrams an improvement on this that effectively avoids the possibility that access to other PSDs connected on the same device-side IO device interconnect might be disrupted due to a malfunction that causes a single device-side interconnect to fail by making use of dual-ported PSDs and adding an additional interconnect to each PSD. In this way, the blockage of a single device-side IO device interconnect, possibly caused by a malfunction of an interconnect controller IC on the PSD, would not result in the inaccessibility of other PSDs connected on the same interconnect because the second interconnect connected to each of the same PSDs can be used to access those PSDs without interference.
The configuration shown in FIG. 2 has the further advantage that IO request load can be distributed between the redundant device-side interconnects thereby effectively doubling the overall bandwidth of the device-side IO device interconnect subsystem as compared to the single-interconnect-per-PSD-set configuration shown in FIG. 1. In this case, the typical device-side IO device interconnect of choice would typically be Fibre FC-AL because of the dual-ported nature of Fibre FC-AL PSDs currently on the market and the elements of the Fibre protocol that allow an initiator, such as an SVC, to determine which interconnect IDs on different interconnects correspond to the same PSD.
While the configuration depicted in FIG. 2 is, indeed, far more robust than that depicted in FIG. 1 in the face of device-side IO device interconnect failure, there is still the possibility that a PSD might malfunction in such a way that it could bring down both IO device interconnects that are connected to its dual-ported port pair. Were this to happen, once again, access to other PSDs connected on the same interconnect pair would be disrupted. In a logical media unit that consists of a standard singly-redundant RAID combination of PSDs (e.g., RAID 5), this could prove disastrous for it can cause multiple PSDs in the combination to go off line causing the entire LMU to go off line.